MEMORANDUM March 3, 2017 


TO: Jharrett M Bryantt 
Assistant Superintendent, College and Career Readiness 


FROM: Carla Stevens 
Assistant Superintendent, Research and Accountability 


SUBJECT: 2015-2016 AVID PROGRAM 


The Advancement Via Individual Determination (AVID) program was developed to increase the 
number of secondary students who participate in rigorous academic courses, to accelerate 
student learning, and to improve student performance. The AVID program in the Houston 
Independent School District (HISD) targets students who (1) are in the academic middle and 
earn grades of B, C, and D; (2) desire to go to college; (3) are willing to work hard; (4) are 
capable of completing rigorous curricula; and (5) are not reaching their full academic potential. 
The program’s mission is to close achievement gaps through the use of educational strategies 
that prepare all students for success. 


Key findings are as follows: 

e Overall, 2,061 HISD students from 23 campuses were enrolled in the 2015-2016 AVID 
program, a 57.3 percent increase from 1,310 in 2014—2015. Student enrollment increased 
76.7 percent at the middle school level and decreased 21.5 percent at the high school level. 

e Nearly one-half of AVID students (46.0 percent) and nearly one-third of non-AVID students 
(30.6 percent) enrolled in pre-Advanced Placement courses. In addition, 22.4 percent of 
AVID students versus 3.4 percent of non-AVID students enrolled in Advanced Placement 
courses and 67.1 percent of AVID students versus 1.5 percent of non-AVID students 
enrolled in dual credit courses. 

e From 2014-2015 to 2015-2016, the percentage of students in AVID who took AP exams 
increased slightly from 20.9 to 21.1, while the percentage who scored three or higher 
decreased 5.5 percentage points. However, a higher percentage of AVID (12.2 percent) 
than non-AVID (8.2 percent) students scored three or higher on AP exams in 2015-2016. 

e Sixth-grade AVID students scored statistically significantly higher (17.3 points) on the 
STAAR reading exam than matched grade six, non-AVID students. A positive, statistically 
significant effect of the AVID program was associated with an average gain of 11.4 points on 
this exam. 

e AVID students achieved their highest pre-AP course grades (grades 6-12) and STAAR 
reading scores (grades 6-8) on Certified or Highly-Certified campuses and performed 
statistically significantly better than AVID students on campuses with lower AVID certification 
levels. 


Further distribution of this report is at your discretion. Should you have any further questions, 
please contact me at 713-556-6700. 


AL CJS 


Attachment 


cc: Grenita Lathan 
Rick Cruz 


ee, — 


HOUSTON INDEPENDENT SCHOOL DISTRICT 


RESEARCH 


Educational Program Report 


ADVANCED VIA INDIVIDUAL 


rs 


~= 


Research and Accountability 


ANALY ZING DATA, MEASURING PERFORMANCE. 


HISD 


2017 BOARD OF EDUCATION 


te ool Wanda Adams 
President 


Diana Davila 
First Vice President 


Jolanda Jones 
Second Vice President 


Rhonda Skillern-Jones 
Secretary 


Anne Sung 
Assistant Secretary 


Anna Eastman 

Manuel Rodriguez, Jr. 
Michael L. Lunceford 
Holly Maria Flynn Vilaseca 


Richard A. Carranza 
Superintendent of Schools 


Carla Stevens 
Assistant Superintendent 
Department of Research and Accountability 


Ngozi J. Kamau, Ph.D. 
Research Specialist 


Lissa Heckelman, Ph.D. 
Research Manager 


Houston Independent School District 
Hattie Mae White Educational Support Center 
4400 West 18th StreetHouston, Texas 77092-8501 


www.HoustonlSD.org 


It is the policy of the Houston Independent School 
District not to discriminate on the basis of age, color, 
handicap or disability, ancestry, national origin, 
marital status, race, religion, sex, veteran status, 
political aff liation, sexual orientation, gender identity 
and/or gender expression in its educational or 
employment programs and activities. 


ADVANCEMENT VIA INDIVIDUAL DETERMINATION (AVID) 
FINDINGS RELATED TO STUDENT PERFORMANCE, 2015-2016 


EXECUTIVE SUMMARY 


Program Description 

The Advancement Via Individual Determination (AVID) program was developed in 1980 to meet students’ 
academic and emotional needs. AVID is used in more than 5,600 schools to help students of color and 
students with socioeconomic disadvantages succeed beyond obstacles that have historically kept their 
ancestors out of college (AVID, 2016a). The program’s mission is to close achievement gaps and prepare 
all students for success in college and in our global society (AVID, 2015). AVID classrooms feature student- 
centered decision-making and learning contracts, academic reading and writing, teachers’ roles as 
students’ advocate/advisor/counselor, academic support from trained teachers and tutors, emphasis on 
objective data, and a commitment to the Socratic process of inquiry. AVID focuses on students who possess 
the capacity to complete college-preparatory coursework with the proper academic and emotional support. 
Equity in education is AVID’s fundamental principle (Magee, 2015). 


The primary objectives of the AVID program are to increase the number of secondary students who 
participate in rigorous academic courses (such as Advanced Placement (AP) courses), to accelerate 
student learning, and to improve student performance. These objectives align with the district's Primary 
Goal 1, “Increase Student Achievement." The AVID program in the Houston Independent School District 
(HISD) targets middle and high school students who (1) are in the “academic middle” and earn grades of 
B, C, and D; (2) desire to go to college; (3) are willing to work hard; (4) are capable of completing rigorous 
curricula; and (5) are not reaching their full academic potential. Typically, these students (1) are enrolled in 
regular classes (non-gifted and talented, non-special education); (2) are economically disadvantaged, (3) 
are from non-White families; (4) are underrepresented in four-year colleges; and/or (5) possess the potential 
to become first-generation college students (Houston Independent School District, 2011). 


To be selected for the AVID program, students must have a GPA between 2.0 and 3.5 and never have 
taken an AP course. HISD teachers who elect to participate in the AVID program are expected to receive 
training on the AVID curriculum to ensure fidelity of implementation. Students who participate in the AVID 
program are expected to take AVID elective courses and receive tutoring from their peers or from volunteer 
AVID tutors, one to two times weekly. Volunteer AVID tutors are expected to receive training and provide 
content-specific support and guidance with reading, study skills, note-taking, organizational skills, writing, 
inquiry, collaboration, and critical thinking including mathematical reasoning to enhance student learning. 
In 2015-2016, students on 23 secondary campuses in HISD participated in the AVID program. 


Highlights 
. e : Student participation in AVID was its highest in 2015-2016, showing a 25.6 percent increase over 
the last four years. Program participation increased 57.3 percent from the 2014—2015 (n=1,310) to 
the 2015-2016 school year (n=2,061), with an increase of 76.7 percent at the middle school level 
and 21.5 percent at the high school level. 


e The 2015-2016 Certification Self-Study (CSS) designations for 22 campuses assessed out of 23 


AVID campuses were five Non-Certified Sites, one Affiliate AVID Site, 11 New AVID Sites, four 
AVID Certified Sites, and one AVID Highly Certified Site (from lowest to highest certification level). 
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e Pre-Advanced Placement (pre-AP), Advanced Placement (AP), and dual credit course enrollment 
rates for students in the AVID program were higher than non-AVID students’ enrollment rates at 
the high school level and in total. 


e Ingrades 10—12, program impact results for matched AVID and non-AVID students showed AVID 
had a positive, statistically significant treatment effect on pre-AP course grades, associated with 
an average gain of 2.4 points due to AVID participation. 


e The percentage of Advanced Placement exams on which AVID students scored three or higher 
(12.2 percent) was 4.0 percentage points higher than the percentage of non-AVID students who 
scored three or higher (8.2 percent) on Advanced Placement exams in 2015-2016. However, the 
proportion of students in AVID scoring three or higher on AP exams in 2015-2016 decreased 5.5 
percentage points from 17.7 percent in 2014-2015. 


e The 17.3 point difference between the mean STAAR reading score of matched grade six AVID and 
non-AVID students was statistically significant in favor of AVID students. A positive, statistically 
significant treatment effect of the AVID program was associated with an average gain of 11.4 points 
on the exam. 


e None of the student performance differences between matched AVID and non-AVID students and 
none of the program treatment effects on the STAAR EOC exam results assessed (English |, 
English Il, U.S. History) were of statistical significance. Sufficient data were not available for 
Algebra | and Biology STAAR EOC exams. 


e The performances of matched AVID and non-AVID students on college readiness exams showed 
statistically significant differences in favor of non-AVID students in mathematics at grades eight 
(PSAT 8/9) and eleven (PSAT NMSQT). Statistically significant, negative program impacts on 
students’ performances in mathematics (grade eight: -7.2 points; grade 11: -53.1 points) and on 
total scores (grade eight: -11.2 points; grade 11: -77.9 points) were found on these measures. 


e AVID students achieved their highest pre-AP course grades (grades 6-12) and STAAR reading 
scores (grades 6-8) on AVID campuses that were Certified or Highly-Certified, outperforming AVID 
students on campuses with lower certification levels on these measures. This result was statistically 
significant (p.<0.5) and small to medium impacts of the AVID program were found. 


Recommendations 
e Establish procedures to ensure that all students targeted by the district for program participation 
are appropriately identified and encouraged to participate in AVID. 


e Ensure that AVID administrators and coordinators in HISD are well-educated through AVID’s 
professional development program and provide support to trained AVID teachers and site teams. 


e Ensure that campus-based, interdisciplinary AVID site teams are afforded the necessary support 
and oversight to guide full program implementation, correctly self-assess adherence to the 11 AVID 
essentials, and accurately report implementation fidelity on their campuses. 


e Further improve AVID students’ performance by identifying gaps in teaching and learning, and 
knowledge retention and transfer; especially among historically underserved student groups. 
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Introduction 


The Advancement Via Individual Determination (AVID) system is implemented in the Houston Independent 
School District (HISD) as a secondary education program for select students. It provides students a platform 
of educational and social structures that allows students to be nurtured through the development of their 
academic skills and motivation to pursue educational and career goals. 


Equity in education is AVID’s fundamental premise (Magee, 2015). The AVID philosophy and framework 
focus on students, particularly low-income students, who possess the capacity to complete college- 
preparatory coursework with the proper academic and emotional support. The program’s mission is to close 
achievement gaps through the use of educational strategies that prepare all students for success in a 
college or university and in our global society (AVID, 2015). AVID utilizes an array of strategies to support 
HISD students’ successful completion of high school and entrance into college. The AVID program employs 
five basic instructional strategies to help students to develop their academic skills for success: writing to 
learn, inquiry to target students’ needs, collaboration, organization, and reading to learn, Known as WICOR. 
Using AVID’s research-based curriculum, reading, writing, critical thinking, and academic behaviors are 
emphasized across subjects to help students clarify, organize, understand, and communicate ideas as well 
as manage their time and set goals (AVID, 2016b). To improve their writing, students may participate in 
Cornell note-taking, prewriting, journaling/learning logs, draft and final draft, editing, and reader response 
activities. Activities to help students become more effective and confident life-long learners and readers 
include survey/question/read/record/recite/review/reflect (SQ5R), What | Know/Want to Learn/Learned 
(KWL), reciprocal teaching, and Think-Aloud. Unlike traditional models for teaching and learning, 
collaborative processes are infused throughout the AVID program and include the teacher as student 
advocate/advisor/counselor, group projects, study groups, Jigsaw Activities, response/edit/revision groups, 
and Read-Around. Inquiry or questioning is AVID’s foundational strategy and is used to help AVID students 
critique and synthesize information so they may advance to higher levels of thinking incrementally. This is 
achieved through skilled questioning and Socratic Seminars and critical thinking and open-mindedness 
activities (Contreras, et al., 2009). AVID students may apply the techniques they learn in the AVID program 
in all their courses. Students on 20 middle and three high school campuses in HISD participated in AVID 
during the 2015-2016 school year (Table 1, p. 29). 


Program Components 
The AVID system components are (AVID, 2013; pp. 3-5): 

e Interdisciplinary site team (AVID professional learning community) creates an annual site plan, 
implements AVID, provides peer support to site team, models best practices, and trains other 
site teachers. 

e District Director oversees and ensures program implementation fidelity. 

e AV/ID-trained teachers deliver academic support and instruction (WICOR). 

e AVID curricular materials and resources, and research-based strategies and best practices. 

e AvVID structure and processes that create a campus-wide attitude of college readiness, produce 
a sense of family among students, and engage parents and the community. 

e AVID professional development in AVID methodologies for educators is exceptional and 
includes district and regional sessions. 

e Ongoing site visits, online seminars, coaching, support and resources are provided for 
teachers. 

e Comprehensive data collection and certification system are implemented to provide districts 
and schools with data to support student improvements. 
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Program Implementation 

AVID program implementation fidelity is determined by a school’s adherence to 11 characteristics of AVID 
secondary school implementation that are considered essential components for maximum impact on 
student success as identified by the AVID Center (AVID, 2016c; p. 1). They are: 


1. AVID student selection must focus on students in the middle, with academic potential, 
who would benefit from AVID support to improve their academic record and begin 
college preparation. 


2. AVID program participants, both students and staff, must participate voluntarily. 


3. The school must be committed to full implementation of AVID, with students enrolled 
in the year-long AVID elective class(es) available within the regular school day. 


4. AVID students must be enrolled in a rigorous course of study that will enable them to 
meet requirements for university enrollment. 


5. Instructional strategies are taught in the AVID Elective class to develop students’ 
organizational skills that promote academic self-management. 


6. Astrong, relevant writing and reading curriculum provides a basis for instruction in the 
AVID elective class. 


7. Inquiry and collaboration are used as a basis for instruction in the AVID elective class 
and to promote critical thinking. 


8. A sufficient number of tutors must be available in the AVID elective class(es) to 
facilitate student access to rigorous curriculum. Tutors should be students enrolled in 
colleges and universities who can mentor students and facilitate tutorials, and they 
must be trained to implement the methodologies used in AVID. 


9. AVID program implementation and student progress must be monitored through the 
AVID Center Data System, and results must be analyzed to ensure success. 


10. The school or district has identified resources for program costs, has agreed to 
implement all AVID Essentials and to participate in AVID Certification. It has committed 
to ongoing participation in AVID professional learning. 


11. An active, interdisciplinary AVID site team collaborates on issues of student access to 
and success in rigorous college preparatory courses. 


Program Certification 

Through a process called Certification Self-Study (CSS), AVID certification categories for participating AVID 
middle and high schools are based upon the schools’ AVID Site Teams’ self-reported levels of 
implementation for each of the 11 AVID Essentials with the districts AVID Director's approval and the 
Team’s submission of AVID's data collection forms. The CSS is used to guide schools toward effective 
program implementation, determine implementation fidelity, and recommend a level of AVID certification 
(AVID, 2016d; Johnston, Nickel, Popp, & Marcus, 2010). The AVID certification categories are: 
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1. Non-Certified Site 
A site that has never been certified and has one or more Essentials rated as “Not 
AVID” (Level 0) and is working to implement all 11 AVID Essentials. 


2. Affiliate AVID Site 
A former AVID Certified Site or Affiliate site that has one or more Essentials rated as 
“Not AVID” (Level 0); Site Team is working to implement all 11 AVID Essentials. 


3. New AVID Site 
A site implementing AVID for the first time at the start of this academic year. 
[Some sites had last enrolled students in AVID electives two or more years prior]. 


4. AVID Certified Site 
A site with all Essentials rated “Meets Certification Standards” (Level 1) or higher. 


5. AVID Highly Certified Site 
A site that meets certification standards (all 11 Essentials at Level 1 or higher), has at 


least six Essentials rated "Routine Use" (Level 2) and/or "Institutionalization" (Level 
3), has been certified for at least two consecutive years, and has implemented for a 
minimum of four years. 


6. Demonstration Site 
A middle school site that has been certified for at least two consecutive years or a high 
school site that has been certified for at least three consecutive years with at least one 
graduating class of seniors; and all 11 Essentials rated "Routine Use" (Level 2) with no 
Essential rated “Not AVID” (Level 0); and the site is recommended as a model of high 
implementation fidelity which is verified through a validation site visit by a qualified 
AVID Center personnel. 


Using data from the CSS, AVID researchers (Johnston, Nickel, Popp, & Marcus, 2010) found that schools 
implementing the AVID program at the highest levels of fidelity achieved significantly higher student 
achievement across all course enrollment and academic outcomes. Students attending AVID Certified 
schools had higher levels of participation in AP/IB courses and on AP/IB exams. They were also more likely 
than their counterparts to take SAT or ACT exams, complete college entrance requirements, and plan on 
attending college. 


The research questions that guided this analysis were: 


e What were the levels of participation in the AVID program from 2012-2013 through 2015-2016? 

e Howdid the characteristics of 2015-2016 AVID participants compare to the characteristics of their non- 
AVID counterparts? 

e What were the levels of AVID program implementation on HISD campuses with students enrolled in 
AVID elective courses? What were the effects of implementation fidelity on AVID student performance? 

e Did AVID students enroll in more pre-AP, AP, and dual credit courses than non-AVID students? 

e How did the levels of AP exam participation and performance of students in the AVID program compare 
between the 2014-2015 and 2015—2016 school years? 

e Did AVID students complete more AP exams and score higher than their non-AVID counterparts? 

e Did AVID students perform better than non-AVID students on state-mandated STAAR and STAAR EOC 
tests and college/career readiness exams? 
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e What accomplishments were achieved by graduating seniors who participated in the AVID program in 
2015-2016, as reported by campus and district staff? 


Methods 


e In late May 2016, student-level Cognos Chancery Ad Hoc Student Information System (Cognos 
Chancery Ad Hoc) records were used to identify students who participated in at least one AVID elective 
course during the 2015—2016 school year. A total of 23 AVID schools (20 middle and three high) were 
identified with 2,061 students (1,502 middle and 559 high) who enrolled in AVID courses. 


e 2015-2016 demographic data were available from Chancery SMS Database on July 7, 2016 (except 
at risk, special education, and gifted/talented data which were retrieved on May 20, 2016 from Cognos 
Chancery Ad Hoc Database) for all students on AVID campuses in grades 6—12. AVID campus of 
enrollment and grade level, PEIMS inclusion (i.e., student was enrolled in HISD on October 30, 2015), 
and AVID course participation data were retrieved from Cognos Chancery Ad Hoc databases on May 
20, 2016. Course participation and completion data were retrieved from Cognos Chancery Ad Hoc 
Grades files on July 7, 2016. 


e AVID participants and all other students on AVID campuses in the same grade levels were matched to 
HISD pre-Advanced Placement (pre-AP), Advanced Placement (AP), and dual credit grades and/or 
examination (exam) databases. The data were used to determine students’ participation and grades in 
advanced courses (on a scale of 0-100), and performance on exams. AP test scores were retrieved 
from Chancery SMS Database Historical Grades File on July 7, 2016. 


e Identified students were matched to their HISD 2015-2016 Public Education Information Management 
System (PEIMS) student information from Cognos Chancery Ad Hoc and to their first administration 
2015-2016 State of Texas Assessments of Academic Readiness (STAAR) scale scores, July 11, 2016 
Release (retrieved on July 27, 2016), spring administration only STAAR End-of-Course (EOC) scale 
scores (retrieved on July 27, 2016) from Cognos Chancery STAAR All Inclusive databases (filtered for 
STAAR, STAAR L, Accommodated, and Alternate 2 test versions), PSAT 8/9 (grades 8 and 9), 
PSAT/NMSQT (grades 10 and 11), and SAT (grades 11 and 12) scale scores (retrieved on September 
7, 2016) from HISD Fall 2015 PSAT89, HISD Fall 2015 PSAT, and HISD 2015-2016 SAT databases. 


e Onescore for each STAAR EOC first time tester or retester was included in analysis. First time STAAR 
EOC testers’ results are based on the Level Il Satisfactory 2016 progression standard which increased 
from the previous Level II Phase-in 1 Satisfactory standard and will continue to increase each year until 
2021-2022. This means first time STAAR EOC test takers had to answer more items correctly to meet 
Level II Satisfactory standard than the items required for retesters of EOC exams in the previous year. 
Retesters had previously taken an EOC exam. Their scores were based on the standard in place at the 
student's first time of testing. To accommodate this variation across students, the STAAR EOC Level 
Il Satisfactory Student Standard field in Cognos Chancery was used. Any comparisons of results to 
prior performance should be made with caution. 


e PSAT 8/9 tests (for grades eight and nine) and PSAT/NMSQT tests (designed for grade 11 and taken 
at grades 10 and 11 in HISD) were taken in October 2015 to measure skills and knowledge for college 
and/or career readiness, with an Evidence-Based Reading and Writing (EBRW) score (reading, 
language, and writing combined) and a mathematics score. PSAT 8/9 EBRW and mathematics scale 
scores range from 120 to 720 points each, with a total score range of 240 to 1440. PSAT NMSQT scale 
scores range from 160 to 760 points, with a total score range of 320 to 1520. 
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e SAT scores are used for college admissions. Scale scores range from 200 to 800 points for each 
subject (critical reading, math, and writing), a maximum total score of 2400. Tests were taken from April 
2015 to January 2016. (The new SAT subjects parallel the new PSAT subjects starting in March 2016.) 


e For performance comparisons, student-level Cognos Chancery Ad Hoc records were used to identify 
8,122 middle and 1,942 high school students who were not enrolled in an AVID elective course in 2015— 
2016 on the same campuses and at the same grade levels as the AVID participants. All students on 
the Houston Academy for International Studies high school campus participated in AVID. Therefore, no 
students were available for the matched, non-AVID comparison group and the campus was excluded 
from analyses that used matched comparisons for student performance. 


e The 2015-2016 students who participated in at least one AVID elective course during the 2015-2016 
school year constituted the treatment group (AVID students). Remaining students at the same 
campuses and grade levels comprised the non-treatment (non-AVID) comparison group. 


e Propensity score matching (PSM) is used to make causal inferences based on observational data 
(Cohen, 1988). To assess the impact of the AVID program on students’ course grades and test 
performances, STATA 14 Treatment-effects using PSM assigns the potential outcome (which is 
considered as missing data) for each AVID student and non-AVID student using an average of the 
outcomes of similar students in the other group (StataCorp, 2013) to estimate program effects. It takes 
into account the estimated nature of propensity scores which are used to depict similarity between 
students based on estimated treatment probabilities. The average treatment effect is derived from the 
average of the difference between the observed and the potential outcome for each student. STATA 
14 Treatment-effects calculates the estimated, average treatment effect among AVID students (ATET). 


e For PSM analyses of students’ course grades, STAAR (reading and mathematics) and STAAR EOC 
(English I, English Il, Algebra I, Biology, and U.S. History) scores, and college readiness scores, AVID 
(treatment group) and non-AVID (comparison group) students were matched on their economic, at-risk, 
special education, gifted/talented, and limited English proficiency statuses, and on their prior year’s 
cumulative GPA (2015-2016 grades 10-12) or spring 2015 STAAR performance (2015-2016 grades 
6—9). Students were also matched on the number of years in which their school participated in the AVID 
program for STAAR and STAAR EOC performance analyses. PSM utilizes a regression model for the 
treatment and requires no bias-correction between the treatment and comparison groups (StataCorp, 
2013, p. 44). PSM analyses generated t-statistics which were used to calculate probabilities (p-values) 
to determine the statistical significance of results. 


e Included in PSM analyses of treatment-effects were students who were enrolled in HISD on October 
30, 2015 (i.e., PEIMS inclusion), had an average daily attendance rate that was greater than zero, and 
who had spring 2015 STAAR scores (2015—2016 grades 6-9) or 2014—2015 cumulative GPA (2015- 
2016 grades 10-12) the previous year. 


e Differences in the educational programming received by AVID students were expected to largely reflect 
the fidelity of AVID program implementation on AVID campuses. IBM SPSS Statistics Kruskal-Wallis 
H-tests for non-random, normally distributed data were employed to ascertain whether or not the 
performances of AVID students differed depending upon the level of program implementation fidelity 
on their campuses (using Certification Self-Study/CSS categories of 1-Non-Certified to 5-Highly 
Certified). Students were grouped by their schools’ level of AVID certification, based on the CSS. AVID 
students who were matched with non-AVID students in previous analyses were included. 
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Kruskal-Wallis H-tests compare three or more groups’ median scores, convert them to mean ranks, 
and evaluate the statistical significance of the difference between the groups. An adjusted significance 
level was used to assure no more than a five percent type-1 error rate for any of the hypotheses being 
tested. Post hoc tests were conducted to determine between which groups statistically significant 
differences were found and to generate z-scores to calculate effect sizes (r) (i.e., Z-score/square root 
of total sample size) (Rosenthal & DiMatteo, 2001; Elis, 2010; Field, 2013), using the following intervals 
for rand Cohen’s d: 0.2 (small effect), 0.5 (medium or intermediate effect), and 0.8 and higher (large 
effect) (Cohen, 1988). 


Kruskal-Wallis H-tests were conducted to compare AVID students’ grades 6—12 mean Pre-AP, AP, and 
dual credit course grades; and grades 6-8 STAAR reading and STAAR EOC English | and English II 
scores grouped by their schools’ AVID certification levels. Sufficient numbers of the students with 2015— 
2016 scores from schools at different levels of AVID program implementation fidelity were not available 
to determine if AVID students’ dual credit course and STAAR EOC performances were impacted by 
their schools’ AVID CSS certification category. 


Where indicated, preliminary analyses were conducted to assure there were no violations of normality, 
linearity, multicollinearity, and homoscedasticity. 


Data Limitations 


The comparison group of non-AVID students was a convenience sample of non-AVID students 
in the same schools and grades, and in many cases, in the same courses with AVID students and 
AVID teachers. AVID teachers who taught core content areas taught both AVID and non-AVID 
students, using the same instructional strategies. Consequently, the extent to which AVID students may 
have experienced instructional strategies and supports that were different from those experienced by 
non-AVID students is undetermined. Therefore, although the comparison group used for the analyses 
was a Statistically matched sample as necessary for rigorous analyses of program effects, while 
correlations can be identified, conclusions of causation are made with caution. 


For some measures, student performance data showing comparisons of AVID and non-AVID 
students were not presented at some grade levels due to group sizes of fewer than five students. 


Data limitations regarding sample size, collinearity, and treatment overlap assumption violations, 
precluded analyses of program effects on average dual credit course grades and on STAAR grade 
seven writing and grade eight science and social studies exams. 


Campus-level AVID implementation data were not available for one middle school. Consequently, the 
school and its students were not included in program implementation analyses. 


All AVID students with dual credit course scores attended schools with the same CSS category (AVID 
Certified Site), which precluded analysis of this measure. 
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Results 


Descriptive Outcomes for Unmatched AVID and Non-AVID Students 


What were the levels of participation in the AVID program from 2012-2013 through 2015-2016? 


In 2015-2016, 1,502 middle and 559 high school students participated in AVID (Figure 1). 


Figure 1 shows AVID program enrollment increased 25.6 percent over the last four years from 2012-— 
2013 (n=1,641) to 2015-2016 (n=2,061), including an 81.4 percent increase among middle school 
students and a 31.2 percent decrease among high school students. Driven largely by middle school 
enrollment, the highest enrollment in the four year time period was in 2015-2016 (Table 1, p. 29). 


From 2014-2015 (n=1,310) to 2015-2016 (n=2,061), there was 57.3 percent growth in AVID program 
participation, which included a 76.7 percent increase among middle school students and a 21.5 percent 
increase among high school students. 


Figure 1. AVID participation, 2012-2013 through 2015-2016 
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Source: Cognos Chancery Ad Hoc, May 20, 2016; Department of Research and Accountability, February 2016 
Figure 2 shows AVID participation increased at each middle school grade level and at grades 10 and 
12 from 2014—2015 to 2015-2016, with the most growth at grade 12 (exceeding 50 times more or 5,850 
percent) and grade six (152.3 percent). Grade 11 had the largest decline (18.6 percent). 


Figure 2. AVID participation by grade level, 2014—2015 and 2015-2016 
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How did the characteristics of 2015-2016 AVID participants compare to the characteristics of their 
non-AVID counterparts? 


e There were 2,061 AVID participants (Table 1, p. 29) and 10,064 non-AVID students (Table 2, p. 30) at 
the same grade levels as AVID students (Table 3, p. 31) across the 23 AVID campuses in HISD in 
2015-2016. 


e To assess the extent to which AVID participation was comparable to non-AVID participation at each 
grade level, Figure 3 shows the proportions of the total number of AVID and non-AVID students by 
grade level. To indicate comparable AVID participation, the proportion of AVID students was expected 
to be similar to the proportion of non-AVID students at the same grade level. 


e The proportions of AVID and non-AVID students were most comparable at grade nine, with a difference 
of 0.7 percentage points. The proportions of AVID students and non-AVID students differed most at 
grade seven by 5.6 percentage points (Table 3). 


Figure 3. Percentage of total AVID and total non-AVID students by grade level, 2015-2016 
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e There were greater proportions of non-AVID than AVID students in grades six, seven, and nine and 
greater proportions of AVID than non-AVID students in grades eight and ten through twelve (Table 3). 


e To assess similarities and differences between AVID students and non-AVID students further, Figure 
4 (p. 11) shows the percentages of AVID and non-AVID students by their demographic characteristics. 


e A total of 93.4 percent of AVID students and 94.0 percent of non-AVID students were Hispanic or 
African American, with far more Hispanic than African American students. 


e A greater proportion of AVID students (85.2 percent) was economically disadvantaged than the 
proportion among non-AVID students (84.0). However, a larger proportion of non-AVID students (71.3 


percent) than AVID students (62.2 percent) was classified as students at-risk. 


e Much larger proportions of LEP (22.1 percent) and special education students (12.9 percent) were 
found among non-AVID than among AVID students (11.3 percent and 3.9 percent, respectively). 


e The proportion of gifted and talented students among AVID participants (13.0 percent) was greater than 
among non-AVID students (10.3 percent) (Figure 4; Table 3). 


HISD Research and Accountability 10 


Percent 
of Students 
NOs 
eC oa ©: 
AK 
oa 
re 
oO 
Ny 
00 
i} 
Hi 
fo} 
ib 
(se 
ND 
Ww 
oo 
| op ° 
[o>) 
wo 
wo 
G 
wo 
wo 
= 
uo 
(=) 
NK 
a 
RB 
= Bs 
He 
wo 
o 
Special Ed L, B 
ie) 
io 
an 
wo 
° 
is 
ic: 
na 


Figure 4. Demographic characteristics of unmatched AVID and non-AVID students, 2015-2016 
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Note: AVID n=2,061; non-AVID n=10,064. Economically Disadvantaged, At-Risk, Special Ed., and Gifted/Talented 
numbers represent duplicated counts across the categories. 


Asian/Pacific Islander 


What were the levels of AVID program implementation on HISD campuses with students enrolled in 
AVID elective courses? 


One-half of the 22 AVID campuses assessed were designated as New AVID Sites, based on their 
2015-2016 Certification Self-Study (CSS) category, while nearly one-fourth of the schools were 
deemed Non-Certified Sites. One-fifth were ranked AVID Certified Sites and met certification standards 
at Level 1-Meets Certification or higher on all 11 AVID Essentials. None of the schools met the highest 
level of implementation fidelity for designation as a Demonstration Site. Implementation data for one of 
the 23 AVID schools were not submitted by its campus-based program administrators to the AVID 
program administrators at its central office (Figure 5; Table 4, p. 32). 


Figure 5. The number of AVID schools rated in each AVID Certification Self-Study (CSS) 
category, 2015-2016 
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The greatest number of the Level 0 (Not AVID) ratings were associated with Essential 8 (Tutors, n=13), 
followed by Essential 2 (Voluntary Participation, n=11) and Essential 9 (Data Collection, n=11). The 
largest number of Level 1 (Meets Certification Standards) ratings were assessed regarding Essential 
10 (Program Cost, Resources, Certification, and Staff Development, n=17), followed by Essential 5 
(Strong Organizational Skills, n=16), and Essential 3 (Full Implementation of the Program, n=15) 
(Figure 6, p. 12; Table 5, p. 33). 


The largest number of the Level 2 (Routine Use) ratings given was three, associated with Essential 3 
(Full Implementation of the Program, n=3), Essential 5 (Strong Organizational Skills, n=3), and 
Essential 6 (Strong Writing and Reading Curriculum, n=3). The greatest number of Level 3 
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(Institutionalization) ratings were given for Essential 4 (Requirements for University Enrollment, n=8), 
followed by Essential 1 (Focus on Students in the Middle, n=1) and Essential 3 (Full Implementation of 
the Program, n=1) (Figure 6; Table 5, p. 33). 


Figure 6. Number of ratings assessed at each level of program implementation for each of the 
11 AVID Essentials on the Certification Self-Study (CSS), 2015-2016 
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Did AVID students enroll in more pre-AP, AP, and dual credit courses than non-AVID students? 


e Figure 7 shows, at the high school level and in total, AVID students enrolled in pre-Advanced 
Placement (pre-AP), Advanced Placement (AP), and dual credit courses at higher rates than their non- 
AVID peers (Tables 6a and 6b, p. 34). 


e =Intotal, nearly one-half of all AVID students (46.0 percent) and one-third of all non-AVID students (30.6 
percent) enrolled in pre-AP courses. In addition, 22.4 percent of all AVID students versus 3.4 percent 
of all non-AVID students enrolled in AP courses and more than two-thirds of high school AVID (67.1 
percent) and a much smaller percentage of non-AVID (1.5 percent) students enrolled in dual credit 
courses (Figure 8). 


Figure 7. Enrollment rates for AVID and non-AVID students in pre-AP, AP, and dual credit 
courses, 2015-2016 
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Source: Cognos Chancery Ad Hoc Student Information System, May 20, 2016; Chancery SMS Database 
Historical Grades File, July 7, 2016. Note: Course enrollment rates are based on the number of AVID or non- 
AVID students who took pre-AP, AP, and dual credit courses and the number of AVID or non-AVID students at 
the school level or in total. 
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Large differences were found between the proportions of AVID students versus non-AVID students 
who enrolled in high school Pre-AP courses (86.9 percent and 20.5 percent, respectively), high school 
AP courses (82.3 percent and 17.1 percent, respectively), and high school dual credit courses (67.1 
percent and 1.5 percent, respectively) (Figure 7, p. 12). 


Though the differences in middle school pre-AP and AP course enrollment were not large, greater 
proportions of non-AVID (33.0 percent and 0.1 percent) than AVID (30.8 percent and <0.1 percent) 
students enrolled. 


Per student course enrollment rates at middle and high school levels and in total show, on average, 
AVID students took 4.2 pre-AP courses, 2.4 AP courses, and/or 3.2 dual credit courses. Non-AVID 
students took of 3.3 pre-AP courses, 2.8 AP courses, and/or 2.0 dual credit courses on average, per 
student (Figure 8; Table 6c, p. 35). These rates are not cumulative across course types for a student. 


Figure 8. Average number of courses per student for AVID and non-AVID students who were 
enrolled in pre-AP, AP, and dual credit courses, 2015-2016 
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Source: Cognos Chancery Ad Hoc Student Information System, May 20, 2016; Chancery SMS Database Historical 
Grades File, July 7, 2016. Note: Per student enrollment rates are based on the number of students who took pre-AP, 
AP, and dual credit courses. Per student rates are not cumulative across course types for a single student. 


Students in AVID had a higher course per student enrollment rate in pre-AP courses at the high school 
level and in total as well as in dual credit courses than did non-AVID students based on the number of 
students who took the courses and the total AVID participants and non-AVID students at each school 
level or in total (Figure 8; Table 6c). 


Non-AVID course per student enrollment rates in AP courses were higher than AVID student rates in 
high school and in total (Figure 8; Table 6c). 


How did the levels of AP exam participation and performance of students in the AVID program 
compare between the 2014-2015 and 2015-2016 school years? 


Figure 9 (p. 14) shows the number of AVID students taking AP exams increased from 274 in 2014— 
2015 to 434 in 2015-2016 (Table 7, p. 35). 


The percentage of AVID participants who took AP exams increased 0.2 percentage points, from 20.9 
percent in 2014-2015 to 21.1 percent in 2015—2016 (Table 7). 


The number of AP exams taken by students in AVID increased by 232 exams (69.5 percent) from 334 
exams in 2014—2015 to 566 exams in 2015-2016 (Figure 9, p. 14; Table 7). 
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Figure 9. Number of students in AVID tested on AP exams and the number of AP exams 
taken by students in AVID, 2014-2015 and 2015-2016 
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Source: HISD AP Exam 2015—16_07-26-16 data file; Department of Research and Accountability, February 2016 


e The number of AP exams on which AVID students scored a three or higher increased 16.9 percent, 
from 59 to 69 exams from 2014—2015 to 2015-2016. However, in the same years, the proportion of AP 
exams on which AVID students scored a three or higher decreased 5.5 percentage points (Figure 10; 
Table 7, p. 35). 


Figure 10. Number and percentage of AP exams on which AVID students scored three or 
higher, 2014-2015 and 2015-2016 
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Source: HISD AP Exam 2015—16_07-26-16 data file; Department of Research and Accountability, February 2016 
Did AVID students complete more AP exams and score higher than their non-AVID counterparts? 


e In 2015-2016, 434 AVID participants took AP exams, which represented 93.9 percent of the 462 AVID 
students who enrolled in AP courses. A lower percentage, 70.1 percent, (239 out of 341) non-AVID 
participants who enrolled in AP courses took AP exams in 2015-2016 (Figure 11; Table 8, p. 35). 


Figure 11. Percentage of students in AVID and their non-AVID peers who took AP exams, 
2015-2016 
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e A total of 434 AVID students completed 566 AP exams for an average of 1.3 AP exams per student 
and 239 non-AVID students completed 366 AP exams, a higher average number of exams taken per 
student (1.5) than taken by AVID students (Figure 12, p. 15; Table 8). 
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Figure 12. Average number of AP exams taken per student by students in AVID and their non- 
AVID peers, 2015-2016 
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e Figure 13 shows that the percentage of AP exams on which students scored three or higher was 4.0 
percentage points greater among AVID than non-AVID students (Table 8, p. 35). 


Figure 13. Percentage of AP exams on which AVID students and their non-AVID peers 
scored three or higher, 2015-2016 
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Did students in AVID receive more Level II and Level Ill Advanced scores on the STAAR and 
STAAR EOC examinations than their non-AVID counterparts? 


e Figure 14 shows AVID students met the Level Il Satisfactory performance standards at higher rates 
than non-AVID students on all STAAR tests administered, except grade seven math and grade eight 
social studies. The largest differences were at grade eight in reading (8.0 percentage points) and math 
(5.8 percentage points) (Table 9, p. 36). 


Figure 14. STAAR Level II Satisfactory 2016 Progression Standard performance for AVID and 
non-AVID students in grades 6-8, 2015-2016 
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Non-AVID students met the Level Ill Advanced performance standard at higher rates than AVID 
students on all STAAR tests administered, except grade seven reading. The largest differences were 
at grade eight in mathematics (3.3 percentage points) and social studies (4.6 percentage points) 
(Figure 15; Table 9, p. 36). 


Figure 15. STAAR Level III Advanced performance for AVID and non-AVID students, 2015-2016 
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Source: Cognos Chancery STAAR All Inclusive All Students, all test versions, STAAR 3-8, July 11, 2016 


STAAR EOC Level II Satisfactory Student Standard results presented in Figure 16 show students 
enrolled in AVID met the Level II performance standards at higher rates than did non-AVID students on 
all five exams. The largest differences were in English | (55.1 percentage points) and English II (61.5 
percentage points) (Table 10, p. 36). 


Figure 16. STAAR End of Course Level II Satisfactory Student Standard performance for 


AVID and non-AVID students, Spring 2016 


98.1 99.1 
87.4 92.6 93.3 


100 
80 
60 


40 


Percent of Students 


English | English II Algebra | Biology U.S. History 


Subject and Grade Level 
mS AVID mNon-AVID 


Source: Cognos Chancery STAAR All Inclusive All Students, all test versions, July 27, 2016 


STAAR EOC Level III Advanced results presented in Figure 17 (p. 17) show students enrolled in AVID 
met Level III Advanced standards at higher rates than did non-AVID students on all five exams. The 
largest differences were in Biology (20.1 percentage points) and U.S. History (20.2 percentage points) 
(Table 10). 
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Figure 17. STAAR End of Course Level III Advanced performance for AVID and non-AVID 
students, Spring 2016 
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Note: STAAR EOC performance results for first time and retesters were included in the analysis. 


What accomplishments were achieved by graduating seniors who participated in the AVID program 
in 2015-2016, as reported by campus and district staff? 


e All graduating seniors who attended Houston Academy for International Studies (HAIS) or Madison 
High School and participated in the AVID program during the 2015-2016 school year completed at 
least one college application. 


e Collectively, the 97 AVID seniors who attended HAIS during the 2015-2016 academic year earned 
more than $6,000,000 in two- and four-year grants and scholarship offers. This represented a 17.5 
percent increase over the $5,104,808 earned by last years’ HAIS seniors in AVID. 


e Collectively, the 22 Madison High School seniors who attended AVID during the 2015-2016 academic 
year and who were eligible to apply to college earned more than $100,000, annually, in two- and four- 
year grants and scholarship offers. 


e More than 50 percent of HAIS graduating seniors (50 out of 97 seniors) earned Associates degrees 
from the Houston Community College in May 2016. Of them, approximately 10 percent achieved the 
Highest Honors. 


e To further their academic studies, HAIS graduates were accepted into Colby College, University of 
Virginia, University of Rochester, Dartmouth, Columbia, Dillard, Jackson State, Hawaii Pacific, and 
others. Of them, three seniors were accepted into the Posse programs at Colby College and the 
University of Virginia. 


e To further their academic studies, Madison High School graduates were accepted into Brandeis 
University, the University of Houston, Texas A&M at Corpus Christi, Lamar University, Texas Southern 
University, the University of Texas, and others. Of them, two seniors were accepted into the Posse 
programs, including one student at Texas A&M. 


e The third AVID high school, Yates, did not have any participants in their senior year of high school. 
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Matched Comparisons of AVID and Non-AVID Student Performance and the Impact of the AVID 
Program AVID Student Performance 


Did students in AVID make higher course grades in Pre-AP, AP, and dual credit courses than their 
non-AVID counterparts? 


e Matched on demographic characteristics and the previous year's performance, results showed the 
differences between AVID and non-AVID students' grades were not of statistical significance as 
measured by their mean pre-AP, AP, and dual credit course grades (Table 11, p. 37). 


e Results for matched AVID and non-AVID students revealed a statistically significant, positive treatment 
effect of the AVID program on the average grades of AVID students taking pre-AP courses in grades 
10-12, as associated with course grades that were an estimated 2.4 points higher (p=.017) (Table 12, 
p. 37). 


e No other findings from the treatment-effects analyses were statistically significant. 


Did students in AVID receive higher mean scores on the STAAR and STAAR EOC examinations 
than their non-AVID counterparts? 


e Matched on demographic characteristics, the previous year's performance, and the number of years of 
campus participation in AVID, results showed a statistically significant difference between STAAR 
reading scores at grade six for matched AVID and non-AVID students, with AVID students scoring 17.3 
points higher than their non-AVID peers (p=.050) (Table 13, p. 38). 


e A Statistically significant, positive treatment effect of the AVID program on the STAAR reading 
performance of grade six AVID students was found. They scored an estimated, average 11.4 points 
higher due to AVID participation (0=.028) (Table 14, p. 38). 


e As previously noted, data limitations regarding sample size, collinearity, and treatment overlap 
assumption violations precluded analyses of program effects on STAAR performance for grade seven 
writing and grade eight science and social studies. 


e STAAREOC scores for first time and retesters were included. Matched on demographic characteristics, 
previous year's performance, and the number of years of campus participation in AVID, results showed 
none of the performance differences between matched AVID and non-AVID students on English |, 
English II, and U.S. History STAAR EOC exams were statistically significant (Table 15, p. 39). 


e Analysis of the program's treatment effects on AVID students' STAAR EOC performances on English 
1, English Il, and U.S. History showed none were of a substantial magnitude (Table 16, p. 39). Data 
limitations regarding sample size, collinearity, and treatment overlap assumption violations precluded 
analyses of program effects on Algebra | and Biology scores. 


Did students in AVID score higher on PSAT 8/9, PSATINMSQT, and SAT exams than their non-AVID 
counterparts? 


e Matched on demographic characteristics and the previous year's performance, results showed a 
statistically significant difference between PSAT 8/9 mathematics scores at grade eight for matched 
AVID and non-AVID students, with non-AVID students scoring 19.9 points higher than their AVID peers 
(p=.028) (Table 17, p. 40). 
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Statistically significant, negative treatment effects of the AVID program on the PSAT 8/9 mathematics 
and total scores of grade eight AVID students were found, as associated with an estimated, average 
7.2 points lower mathematics score (p=.035) and 11.2 points lower in total scores (p=.022) (Table 18, 
p. 40). 


Matched on demographic characteristics and the previous year's performance, results showed a 
statistically significant difference between PSAT NMSQT mathematics scores at grade 11 for matched 
AVID and non-AVID students, with non-AVID students scoring 55.8 points higher than their AVID peers 
(Table 19, p. 41). 


Statistically significant, negative treatment effects of the AVID program on PSAT NMSQT mathematics 
and total scores of grade 11 AVID students were found, as associated with an estimated, average 53.1 
points lower mathematics score (p=.030) and 77.9 points lower in total scores (p=.047) (Table 20, p. 
41). 


Matched on demographic characteristics and the previous year's performance, results showed the 
differences between AVID and non-AVID students' SAT scores were not of statistical significance. No 
substantial program impacts were found regarding the SAT exam (Table 21 and Table 22, p. 42). 


Summary 


Sixty-nine percent (n=20) of the 29 preliminary, unmatched comparisons showed AVID students scored 
higher than non-AVID students across measures. The differences were statistically significant (p.<.05) 
in 60.0 percent (n=12) of the 20 unmatched comparisons in which AVID students scored higher than 
their non-AVID peers. Sufficient data were not available for Algebra | and Biology STAAR EOC exams. 


Matched comparisons showed AVID students scored higher than non-AVID students in 55.2 percent 
(n=16) of the 29 comparisons across measures. On only one measure, the grade six STAAR reading 
exam, was the performance difference statistically significant (p.<.05). This constituted 6.3 percent of 
the 16 matched comparisons in which AVID students scored higher than their non-AVID peers. 
Similarly, on only one measure, the grade 11 PSAT NMSQT exam, was the performance difference in 
favor of non-AVID students and statistically significant (p.<.05). This constituted 7.7 percent of the 13 
matched comparisons in which non-AVID students scored higher than their AVID peers. 


Analyses of program effects found that AVID had positive impacts on AVID student performance in 
55.2 percent (n=16) of the 29 comparisons across measures, but, on only two measures was the impact 
statistically significant (p.<.05), the grade six STAAR reading exam and the grades 10-12 pre-AP 
course grades. This constituted 12.5 percent of the 16 analyses that found favorable AVID impacts. 


On the other hand, analyses of program effects estimated AVID had negative impacts on AVID student 
performance in 44.8 percent (n=13) of the 29 comparisons across measures. The negative impact was 
statistically significant (p.<.05) on four measures, the grade eight PSAT 8/9 mathematics and total score 
and on the PSAT NMQST grade 11 mathematics and total score. This constituted 30.8 percent of the 
13 analyses that found unfavorable AVID impacts, which is more than twice the estimated amount of 
favorable AVID impacts. 
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What were the effects of AVID implementation fidelity on AVID student performance? 


Course Grades 


To determine if AVID students’ performances differed depending upon the level of their campuses’ 
program implementation fidelity, the mean ranks of AVID students’ grades 6-12 pre-AP, AP, and dual 
credit course grades were compared based on their schools’ AVID CSS certification category. 


The comparison of students’ mean pre-AP course grades revealed a statistically significant difference 
in the mean ranks of AVID students’ pre-AP grades between schools with different certification levels, 
(x?(4)=24.890, p<0.001) (Figure 18). 


Figure 18. Comparison of AVID campuses’ mean ranks based on their students’ pre-AP 
course grades by campuses’ AVID Certification Self-Study (CSS) category, 
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Source: Cognos Chancery Ad Hoc Historical Grades File, July 7, 2016. 
Note: None of the students were on Highly-Certified campuses. Bold horizontal bars show mean ranks. 


The mean ranks presented in Figure 18 show AVID students’ pre-AP performance was best on Highly- 
certified campuses (591.72). However, their performance was higher on non-Certified campuses 
(523.85) than on more highly certified campuses (New AVID (468.05), Certified (436.18), and Affiliate 
(351.44)); with students on Certified campuses earning lower grades than students on New AVID 
campuses. 


The difference in AVID students’ pre-AP course grades between AVID schools with Affiliate versus 
Highly-Certified certification levels was statistically significant (p<0.05, r=0.46), indicative of a favorable, 
intermediate program effect. 


Statistically significant differences were found in AVID students’ pre-AP course grades between schools 
with Certified and Highly-Certified (p<0.05, r=0.14) AVID certification levels. However, the program 
effects were small. 


There were also statistically significant differences in AVID students’ pre-AP course grades between 
schools with Affiliate and Non-Certified (p<0.05, r=0.28) and Certified and Non-Certified (p<0.05, 
r=0.12) AVID certification levels, indicating small program effects. 


No statistically significant performance differences were observed in AP course grades between 
students on campuses with different certification levels. 
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There were insufficient numbers of cases at different levels of AVID program implementation fidelity to 
determine if AVID students’ dual credit course grades were impacted by their schools’ AVID certification 
category. 


STAAR Scores 


To determine if AVID students’ performances differed depending upon the level of their campuses’ 
AVID program implementation fidelity, the mean ranks of AVID students’ STAAR scale scores were 
compared based on their schools’ AVID CSS certification category (Figure 19). 


Figure 19. Comparison of AVID students’ average mean ranks for grade six STAAR 
reading scale scores by campuses' AVID certification level, 2015-2016 
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Source: Cognos Chancery Ad Hoc Historical Grades File, July 11, 2016. 
Note: None of the students were on Highly-Certified campuses. Bold horizontal bars show mean ranks. 


A comparison of students’ mean scale scores revealed a statistically significant difference in the mean 
ranks of STAAR reading scores of grade six AVID students between schools with different certification 
levels, (x2(3)=9.848, p<0.05) (Figure 19). 


The mean ranks presented in Figure 19 show grade six AVID students’ STAAR reading performance 
was best on Certified (238.67) and New AVID (195.17) campuses versus the campuses with lower 
certification status. 


AVID students’ performance on non-Certified campuses (193.59) was better than their performance on 
Affiliate campuses (158.34) where AVID students scored lowest. 


The difference in grade six AVID students’ STAAR reading scores between AVID schools with Affiliate 
versus Certified certification levels was statistically significant (p<0.05, r=0.30), indicative of a favorable, 
intermediate program effect. 


A comparison of the mean ranks of grade seven AVID students’ STAAR reading scale scores grouped 
by their schools’ AVID CSS certification category revealed a statistically significant difference in the 
mean ranks of STAAR reading scores of grade seven AVID students between schools with different 
certification levels, (x2(3)=13.219, p<0.05) (Figure 20, page 22). 
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Figure 20. Comparison of AVID students’ average mean ranks for grade seven STAAR 
reading scale scores by campuses' AVID certification level, 2015-2016 
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Source: Cognos Chancery Ad Hoc Historical Grades File, July 11, 2016 
Note: None of the students were on Affiliate campuses. Bold horizontal bars show mean ranks. 


e The mean ranks presented in Figure 20 show grade seven AVID students’ STAAR reading performance 
was best on Certified (207.77) and Highly-Certified (179.91) campuses versus the campuses with lower 
certification levels. However, their performance was better on Certified than on Highly-Certified 
campuses and better on non-Certified (168.24) than on New AVID campuses (151.07), where AVID 
students scored lowest. 


e The difference in AVID students’ grade seven STAAR reading scores between AVID schools with New 
AVID and Certified certification levels was statistically significant (p<0.05, r=0.24), indicative of a 
favorable, but small program effect. 


Figure 21. Comparison of AVID students’ average mean ranks for grade eight STAAR 
reading scale scores by campuses' AVID certification level, 2015-2016 
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Source: Cognos Chancery Ad Hoc Historical Grades File, July 11, 2016. 
Note: None of the students were on Affiliate campuses. Bold horizontal bars show mean ranks. 
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A comparison of the mean ranks of grade eight AVID students’ STAAR reading scale scores grouped 
by their schools’ AVID CSS certification category revealed a statistically significant difference in the 
mean ranks of students’ STAAR reading scores between schools with different certification levels, 
(x2(3)=22.512, p<0.05) (Figure 21, page 22). 


The means presented in Figure 21 show grade eight AVID students’ STAAR reading performance was 
best on Certified (261.95) and Highly-Certified (243.24) campuses versus the campuses with lower 
certification ratings (Non-Certified, 182.76 and New AVID, 196.03). In addition, students’ performance 
was better on New AVID (196.03) than on Non-Certified campuses (182.76) where AVID students 
scored lowest. 


The differences in AVID students’ grade eight STAAR reading scores between AVID schools with Non- 
Certified and Highly-Certified certification levels (p<0.05, r=0.21) and between AVID schools with New 
AVID and Certified certification levels (p<0.05, r=0.24) were statistically significant, indicative of a 
favorable, but small program effects. 


There were statistically significant differences in AVID students’ grade eight STAAR reading scores 
between schools with Non-Certified and Certified (p<0.05, r=0.30) AVID certification levels, indicative 
of a favorable, intermediate effect of the program. 


STAAR EOC Scores 


There were insufficient numbers of cases at different levels of AVID program implementation fidelity to 
determine if AVID students’ performances were impacted by their schools’ AVID certification category. 


Summary 


Across measures, AVID students on AVID campuses that were Certified or Highly-Certified achieved 
the highest mean ranks based on their course grades and exam scale scores, when compared to the 
mean ranks attained by AVID students on campuses with lower AVID certification levels. 


Only in pre-AP courses did AVID students on Non-Certified AVID and New AVID campuses attain the 
second- and third-highest mean ranks, exceeding the rank of AVID students on Certified campuses. 


Statistical significance (p<0.5) was found among AVID students who achieved higher pre-AP course 
grades (grades 6—12) and STAAR reading scores (grades 6-8) on AVID campuses that were Certified 
or Highly-Certified. The results are indicative of small to medium impacts of the AVID program. 


There were no statistically significant performance differences in AP course grades between students 
on campuses with different certification levels. 


Data at different levels of AVID program implementation were insufficient to determine if AVID students’ 
dual credit course and STAAR EOC performances were impacted by their schools’ AVID certification. 
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Discussion 


The district has implemented the AVID program to improve equity and close achievement gaps by targeting 
students who (1) are in the academic “middle” and earn grades of B, C, and D; (2) desire to go to college; 
(3) are willing to work hard; (4) are capable of completing rigorous curricula; and (5) are not reaching their 
full academic potential. Typically, these students (1) are enrolled in regular (non-gifted/talented, non-special 
education) classes; (2) are economically disadvantaged or are from non-White families; (3) are 
underrepresented in four-year colleges; and (4) possess the potential to become first-generation college 
students (Houston Independent School District, 2011). 


Student participation in the AVID program increased 25.6 percent over the last four years. Its highest 
enrollment was in 2015-2016, with a 57.3 percent increase from 2014-2015 (n=1,310) to 2015-2016 
(n=2,061). This included increases of 76.7 percent among middle school students and 21.5 percent among 
high school students. From 2013-2014 to 2014-2015, AVID participation decreased 10.4 percent, including 
a 45.6 percent decrease at the high school level due to changes in AVID participation which resulted in five 
2013-2014 AVID high schools that for various reasons did not have their contracts for participation renewed 
in 2014-2015. The federal Race To The Top grant (managed through the Linked Learning Department) 
began funding middle school students’ participation in AVID in 2014—2015, which resulted in a 37.8 increase 
at the middle school level from 2013-2014 to 2014-2015. In addition, this grant has been largely 
responsible for middle school enrollment almost doubling since the 2014—2015 school year. 


To further support program participation, it may prove beneficial to ensure that all students who meet the 
criteria targeted by the district for program participation are appropriately identified and encouraged to 
participate in AVID. Based on the disproportionality of AVID students versus non-AVID students in 2015— 
2016, continued focus on the inclusion of African American and economically disadvantaged students is 
warranted. In addition, heightened attention to the recruitment of Hispanic and American Indian students, 
students at-risk, and English language learners, particularly at grades six, seven, and nine is encouraged. 


The levels of program implementation on AVID campuses reveal the need for improvement in program 
fidelity. Only 72.7 percent of the 22 “AVID” campuses assessed were designated as AVID sites (n=16), 
based on their 2015-2016 AVID Certification Self-Study (CSS) categories of New AVID, Certified, and 
Highly-Certified AVID sites. None of the schools met the highest level of implementation fidelity to be 
designated a Demonstration Site. The gaps in program fidelity may well indicate deficits in AVID services 
on HISD campuses and may account for some of the disparate results of the matched comparison 
analyses, as discussed later in this section. 


Findings of this report show higher total pre-AP, AP, and dual credit course enrollment rates for AVID than 
for non-AVID students. When AVID and non-AVID students were matched to create comparable groups for 
rigorous statistical analyses, achievement results were mixed. None of the performance differences 
between matched AVID and non-AVID students in pre-AP, AP, and dual credit courses were of statistical 
significance. However, the results showed AVID had a positive, statistically significant treatment effect on 
pre-AP course grades in grades 10-12, associated with an average gain of 2.4 points due to AVID 
participation. 


Analyses in this report also revealed AVID students achieved higher pre-AP course grades (grades 6-12) 
and STAAR reading scores (grades 6-8) on AVID campuses that were Certified or Highly-Certified than 
AVID students attained on these measures on campuses with lower AVID certification levels. Statistically 
significant performance differences between campuses of different certification levels revealed small to 
medium effects of the AVID program, on these measures. 
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In light of research that found the highest levels of AVID certification produced higher student achievement 
in course enrollment and other academic outcomes (Johnston, Nickel, Popp, & Marcus, 2010), the disparate 
findings of this report support the need to institute instructional improvements (including AVID’s strategies) 
to enhance supports for student learning and achievement in all AVID students’ courses. In turn, improved 
program implementation fidelity should boost AVID students' performance in pre-AP, AP, and dual credit 
courses as well as on all state-mandated assessments, because implementation fidelity moderates the 
relationship between interventions and their intended outcomes (Elliot & Mihalic, 2004). 


From 2014-2015 to 2015-2016, the percentage of AVID students taking AP exams increased only 0.2 
percentage points. However, the percentage of exams on which AVID students scored three or higher 
decreased 12.2 percentage points in the same period. Nonetheless, a greater proportion of AVID students 
who enrolled in AP courses took AP exams (93.9 percent) and scored three or higher (12.2 percent) than 
did non-AVID students (70.1 percent and 8.2 percent, respectively) in 2015-2016. To a degree, these 
results are consistent with prior research showing AVID students’ academic participation and outcomes are 
greater than their non-AVID peers (Johnston, Nickel, Popp, & Marcus, 2010; Murray, 2012). However, 
proactive steps to identify gaps in teaching and learning in pre-AP, AP, and dual credit courses may help 
further improve AVID student performance in all areas, including advanced coursework and exams. 


Previous research also indicated students in an AVID program show higher mean scores than non-AVID 
students on state-mandated assessments of reading, mathematics, and science (Murray, 2012). 
Preliminary, descriptive, and statistical analyses conducted for this report showed favorable AVID results 
diminished with greater analytical rigor (i.e., using matched samples of AVID and non-AVID students). 
Specifically, while AVID students achieved higher mean scores than non-AVID students in five of the nine 
matched comparisons of state-mandated tests scores, the results were statistically significant in only one 
case (i.e., grade six AVID students scored an average 17.3 points higher on STAAR reading exams than 
their non-AVID peers). On the same exam, an estimated positive treatment effect of the AVID program 
showed an average gain of 11.4 points due to AVID participation. However, none of the performance 
differences between matched AVID and non-AVID students on STAAR EOC exams (English |, English II, 
and U.S. History) were of statistical significance and none of the program effects were substantial. 


In 2015-2016, the performances of matched AVID and non-AVID students on college readiness exams 
showed statistically significant differences in favor of non-AVID students on mathematics and total scores 
at grades eight (PSAT 8/9) and eleven (PSAT NMSQT). The AVID program was found to have statistically 
significant, negative impacts on students’ performances on these measures. However, no significant 
performance differences or program impacts were found regarding the SAT exam. 


Analyses of the matched performances of AVID students across the assessed standardized measures 
highlight crucial areas for program improvements, as confirmed by results of the program effects on AVID 
student performance. A study in which cognitive principles were applied to education interventions 
concluded that AVID’s conventional strategies (unlike metacognitive approaches) do not support long-term 
retention and transfer of knowledge. Some researchers have suggested this may explain AVID’s failure to 
demonstrate strong program effects on students’ learning and study skills (Education Week, 2015). 
Therefore, it may prove beneficial for AVID administrators to explore ways to improve student learning and 
retention to heighten the academic performance of AVID students. 


Moreover, to better prepare more students in AVID for greater levels of success in advanced courses and 
exams, state-mandated assessments, and college-readiness exams, program administrators should 
consider confirming AVID campuses’ program adherence to the 11 factors that AVID has identified to guide 
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the overall program philosophy, successful implementation of the program, and the success of AVID 
students (Contreras, et al., 2009). Careful attention to the program features may prove beneficial to improve 
program participation and fidelity, as well as heighten positive student achievement outcomes among 
students in the AVID program. It is recommended that AVID administrators in HISD closely examine the 
extent to which AVID schools and teachers in HISD implement each of the 11 AVID Essential factors, which 
include heightened professional development of all AVID stakeholders, employing the AVID curriculum 
without exception, and providing greater instructional supports, as necessary, for AVID students to be as 
successful as possible in all courses and on all examinations. 


As indicated in the Data Limitations section, it is important to note that in HISD, AVID teachers also teach 
non-AVID students and use the same instructional strategies as used with AVID students. This approach 
is consistent with AVID’s new, broader mission of whole school reform (Magee, 2015). However, this 
practice may be indicative of a form of ‘contamination’ of the comparison group through a ‘bleed-over’ of 
treatment. Therefore, analyses of program effects included in this report should be considered with caution. 
For increased rigor, future analyses of program effects may include statistical matching of AVID students 
with students on comparable, non-AVID campuses. 
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Table 1. Number of AVID participants by grade level and school, 2015-2016 
School Name Grade Level 


Burbank MS 7 68 - - 

Clifton MS 31 - = - s 7 - 
Cullen MS 118 135 125 - 7 2 z 
Deady MS 46 18 85 - - “ 
Dowling MS - - 46 - = : - 
Edison MS 41 - = - - & : 
Fleming MS 27 - - : : - 
Fondren MS - - 59 - : “ - 
Fonville MS 31 - : : ‘ 7 z 
Henry MS 29 - - - - - 
Holland MS 32 - - - 7 , = 
Jackson MS 29 24 40 - : z z 
McReynolds MS 32 *L - 2 : : - 
Ortiz MS 44 51 69 - ss 7 z 
Revere MS = 55 46 - : - - 
Stevenson MS 22 18 22 - z : z 
Thomas MS - 17 47 - a Z 
Welch MS 32 *2 - 2 i = : 
West Briar MS 19 14 - 7 : : P 


Williams MS 


Houston Academy for 
International Studies HS* 


Madison HS - - - 18 31 27 22 
Yates HS 


- - - 118 127 “%90 = *%97 


Source: Cognos Chancery Ad Hoc Student Information System, May 20, 2016 

Note: -Indicates no AVID students were enrolled. *For grade levels with fewer than five AVID students, non- 
AVID students were not included in the comparison group. “All students on the Houston Academy for 
International Studies high school campus participated in AVID. Therefore, no non-AVID students were 
available for the comparison group and HAIS students were excluded from analyses that used matched 
comparisons. (See non-AVID student counts in Table 2, p. 30.) 


HISD Research and Accountability 29 


M10) (-wam (Ulan) ol-yare)macelaey-NVA1P My ie lol-1a) cwr-l ma ANV4| DIYer avoto)t-w-laleme | ¢-(e(-M(-\V(-)[-u40 Roped 0M Mo) 


School Name Grade Level 

6th 7th 8th 9th 10th 11th 12th 
Burbank MS - 408 - - : 2 
Clifton MS 246 - - - - - 
Cullen MS 75 93 101 z - : 3 
Deady MS 169 199 181 - : : 
Dowling MS = - 325 - - = : 
Edison MS 177 - - - . : é 
Fleming MS 134 - - : 7 : - 
Fondren MS 7 - 202 - . : 
Fonville MS 255 - - : : 2 
Henry MS 247 - - : 2 2 : 
Holland MS 190 - - - < : 2 
Jackson MS 212 280 263 - : : : 
McReynolds MS 171 * - 2 = 7 : 
Ortiz MS 294 287 280 - 3 : : 
Revere MS - 376 388 - : - 7 
Stevenson MS 400 419 437 - - - - 
Thomas MS - 160 129 - : ‘ Z 
Welch MS 239 : - : z é 
West Briar MS 326 317 - - 


Williams MS 


Houston Academy for 
International Studies HS* 


Madison HS - - - 618 369 329 368 
Yates HS 


Source: Cognos Chancery Ad Hoc Student Information System, May 20, 2016 
Note: -Indicates no AVID students were enrolled, therefore, non-AVID students were not included in the 
comparison group.*For grade levels with fewer than five AVID students, non-AVID students were not included in 
the comparison group. “All students on the Houston Academy for International Studies high school campus 
participated in AVID. Therefore, no students were available for the matched, non-AVID comparison group and 
the campus was excluded from analyses that used matched comparisons for student performance. (See AVID 
student counts in Table 1, p. 29.) 
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Table 3. Grade levels and characteristics of AVID participants and non-AVID 


participants at AVID schools, 2015-2016 


AVID Students Non-AVID Students 
(N=2,061) (N=10,064) 

N % N % 
Grade 
6 560 27.2 3,277 32.6 
7 403 19.6 2,539 25.2 
8 539 26.2 2,306 22.9 
9 164 8.0 876 8.7 
10 158 7.7 369 3.7 
11 118 5.7 329 3.3 
12 119 5.8 368 3.7 
Total 2,061 100.0 10,064 100.0 
Gender 
Male 930 45.1 5,399 53.6 
Female 1,131 54.9 4,665 46.4 
Total 2,061 100.0 10,064 100.0 
Race/Ethnicity 
Asian/Pacific Islander 57 2.8 209 2.1 
American Indian/Alaska Native 3 0.1 18 0.2 
African American 796 38.6 2,647 26.3 
Hispanic 1,130 54.8 6,816 67.7 
White 64 3.1 336 3.3 
Two or more 11 0.5 38 0.4 
Total 2,061 100.0 10,064 100.0 
Economic Disadvantaged 1,755 85.2 8,456 84.0 
At-Risk 1,281 62.2 7,179 71.3 
LEP 233 11.3 2,227 22.1 
Special Education 80 3.9 1,301 12.9 
Gifted/Talented 268 13.0 1,041 10.3 


Source: Cognos Chancery Ad Hoc Student Information System, May 20, 2016; Chancery SMS 
Database on July 7, 2016 

Note: Economic Disadvantaged, At-Risk, Special Ed., and Gifted/Talented numbers represent 
duplicated counts. 
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program implementation, 2015-2016 


Certification Self-Study — Implementation Level 
AVID 
Years Non- = Affiliate New AVID Highly 
in Certified AVID AVID- Certified Certified Demonstration 
School Name AVID Site Site Site Site Site Site 
Burbank MS 2 x 
Clifton MS 1 x 
Cullen MS 1 Xx 
Deady MS 2 Xx 
Dowling MS 1 Xx 
Edison MS 1 - - - - - 
Fleming MS 1 Xx 
Fondren MS 1 Xx 
Fonville MS a x 
Henry MS 4 x 
Holland MS 4 Xx 
Jackson MS 3 Xx 
McReynolds MS 3 x 
Ortiz MS 4 Xx 
Revere MS 4 x 
Stevenson MS 2 x 
Thomas MS 2 x 
Welch MS 2 Xx 
West Briar MS 2 x 
Williams MS 1 x 
Houston Academy for 
International Studies HS 4 x 
Madison HS 4 x 
Yates HS 3 x 
AVID Total Sea a ea eS ae 
Non-Certified Site Affiliate AVID Site New AVID Site 

A site that has never been certified and A former AVID Certified Site or Affiliate site that has one | A site implementing AVID for 

has one or more Essentials rated as “Not | or more Essentials rated as “Not AVID” (Level 0). Site the first time at the start of 
AVID” (Level 0) and is working to Team is working to implement all 11 AVID Essentials. this academic year. 
implement all 11 AVID Essentials. 
AVID Certified Site AVID Highly Certified Site 
A site with all Essentials rated “Meets Certification A site that meets certification standards (all 11 Essentials at Level 1 or 
Standards” (Level 1) or higher. higher), has at least six Essentials rated "Routine Use" (Level 2) and/or 


"Institutionalization" (Level 3), has been certified for at least two consecutive 
years, and has been implementing for a minimum of four years. 


Sources: 2015-2016 AVID Secondary Certification Entry System (online), May 23, 2016; Department of 
Research and Accountability, May 2014, June 2015, and February 2016 (historical data for the number of years 
campuses have participated in AVID) 

Note: *Indicates the number of years sites implemented AVID in the last four years. No data were available for 
Edison MS. 
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Table 5. Certification Self-Study levels for AVID Essentials by number and AVID campus, 


2015-2016 
Certification Self-Study 
Schoot Name AVID Essential Ratings 
(n=23) 1 2 3 4/5 6 7 8 9 10 | 11 | Total 
Burbank MS 0 0 0 1 0 0 0 0 0 0 0 
Clifton MS 1 0 1 1 1 1 0 0 0 1 1 
Cullen MS 1 1 2 0 1 1 1 1 1 1 1 
Deady MS 0 0 0 1 0 0 0 0 0 0 0 
Dowling MS fl 1 1 iL 1 1 1 0 1 1 1 
Edison MS - - - - - - - - - - - 
Fleming MS 1 0 uf 3 1 1 1 1 1 1 1 
Fondren MS 1 0 1 0 41 HE 0 1 0 1 0 
Fonville MS 1 0 1 3 1 0 0 0 0 1 0 
Henry MS 1 1 1 3 1 1 1 1 1 1 1 
Holland MS 0 0 0 1 0 0 0 0 0 0 0 
Jackson MS 0 0 1 0 1 1 1 0 HE 1 1 
McReynolds MS 1 1 1 3), 1 )1 1 0 1 1 HT 
Ortiz MS 1 1 1 1 1 1 1 1 1 1 1 
Revere MS 2 2 2 1 2 2 2 2 2 2 2 
Stevenson MS 1 1 1 a 1 a 1 0 0 1 0 
Thomas MS 0 0 1 0 1 0 0 0 0 1 0 
Welch MS 0 1 1 3 1 ut 1 0 0 1 1 
West Briar MS 0 0 1 3 1 uf 1 0 0 0 0 
Williams MS 0 0 1 3 1 HE 1 0 0 1 1 
Houston Academy for 
International Studies HS 3 1 3 3 | 2 | 2 1 1 2 1 1 
Madison HS 1 1 2 2 2 2 2 1 1 1 1 
Yates HS 0 ul 1 ue 1 1 1 ul 1 1 0 
AVID Total - Level 0 9 11 3 4 | 3 5 7 | 13 | 11 4 9 79 
AVID Total - Level 1 11 | 10 | 15 | 9 | 16|14| 13 | 8 9 17 | 12 134 
AVID Total - Level 2 1 1 3 1 3 3 2 1 2 1 1 19 
AVID Total - Level 3 1 0 1 8 0 0 0 0 0 0 10 
Essential 1 Essential 2 Essential 3 
Focus on Students in the Middle Voluntary Participation Full Implementation of the Program 
Essential 4 Eecantials Essential 6 
Requirements for University ae . Strong Writing and Reading 
Strong Organizational Skills ; 
Enrollment Curriculum 
Besental y : Essential 8 Essential 9 
Collaboration and Inquiry to Tutors Data Collection 
Promote Critical Thinking 
Essen se Essential 11 
Program Cost, Resources, Certification, and Staff Active AVID Site Team 
Development 
Source: 2015-2016 AVID Secondary Certification Entry System (online), May 23, 2016 
Note: No data were available for Edison MS. AVID Essentials are described on page 6 of this report. 
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Milo) (-Mor-Wm Ulan) el-) are) m-N | Pr-lale male)atyANVAl DM-ycUlol-yalcm-valce)i(-(e MoM ol (oy oe emmr- lace melUr-lmela-vellt 
courses by course type and school level, 2015-2016 
Pre-AP AP Dual Credit 


Non- Non- Non- 
AVID AVID AVID AVID AVID AVID 
Middle School 
Students 463 2,678 2 8 : . 
High School 
Students 486 399 460 333 375 29 


Milo) (-M a) oMm Ulin] ol-1me) me) c-ty-Veew-\ oom lavemolUc-\medgclollmexelela-y-M-Jal coli lai(-)alewce) ayANVd| Dir-lavemale)aby-Na |B) 
students by course type, school level, and school, 2015-2016 


Pre-AP AP Dual Credit 

Non- Non- Non- 
oe AVID ae AVID les AVID 

Middle School 
Burbank MS 184 620 - - - - 
Clifton MS 26 403 - : : : 
Cullen MS 57 5 - 7 : 
Deady MS 44 519 - - “ 2 
Dowling MS 202 160 - - - - 
Edison MS 112 164 - - 3 : 
Fleming MS 1 56 - - < : 
Fondren MS 9 33 - - : 
Fonville MS 15 136 - - - - 
Henry MS 36 282 - : : 
Holland MS 90 217 - - “ : 
Jackson MS 21 589 - - - - 
McReynolds MS 8 128 - - - - 
Ortiz MS 321 658 - - “ : 
Revere MS 108 658 4 16 - - 
Stevenson MS 158 2,037 - - - - 
Thomas MS 17 193 - - - - 
Welch MS - 162 - - “ 
Westbriar MS 48 1,575 - - 7 - 
Williams MS 2 39 - = “ Z 


Middle School Total 
Course Enrollments 


High School 

Houston Academy 

for International HS 2,101 - 931 - 1,209 - 
Madison HS 263 1,429 152 931 9 58 


Yates HS 122 113 1 2 - - 


Total Course 
Enrollments 3,945 10,176 1,088 949 1,218 58 


Source: Cognos Chancery Ad Hoc Student Information System, May 20, 2016; Chancery SMS Database Historical 
Grades File, July 7, 2016 
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Table 6c. AVID and non-AVID students’ average per student enrollment rates in pre-AP, AP, 


and dual credit courses by course type and school level, 2015-2016 


Pre-AP AP Dual Credit 
Non- 
AVID Non-AVID AVID Non-AVID AVID AVID 
Middle School 
Students 463 2,678 2 8 - - 
High School 


Students 


Middle School 
Course Enrollments 1,459 8,634 4 16 - - 
High School 

Course Enrollments 2,486 


ource: Cognos Chancery Ad Hoc Student Information System, May 20, 2016; Cha 

Grades File, July 7, 2016 

Note: The average per student enrollment rates were calculated by dividing the number of course enrollments at 
each school level and in total by the number of students at each school level and in total. 


Table 7. AP examination participation and performance results for AVID students, 


2014-2015 and 2015-2016 


2014-2015 2015-2016 
(N=1,310) (N=2,061) Change 
N % N % 

AVID Students Taking Exams 274 20.9 434 21.1 0.2 ppts. 
Exams Taken 334 - 566 - 232 exams 
Exams Scored Three or Higher 59 17.7 69 12.2 -5.5 ppts. 
Average Number of Exams 
per student 1.2 - 1.3 - 0.1 exams 


Sources: HISD AP Exam 2015—16_07-26-16 data file; Department of Research and Accountability, February 2016 
Note: * “ppts.” means percentage points. 


Bile) (<em> Clan iiarcliceyamm ey-laited| ox-uilolamr-lavem el ace aiit-laver-mc-s-101| mre) a-N | Dir- Vale Mm alelaeyNVAID) 


students at AVID schools who enrolled in AP courses, 2015-2016 


AVID Participants Non-AVID Students Enrolled 
Enrolled in AP Courses in AP Courses 
(N=462) (N=341) 
N % N % 
AP Students Taking Exams 434 93.9 239 70.1 
Exams Taken 566 - 366 - 
Exams Scored Three or Higher 69 12.2 30 8.2 
Average Number of Exams 
per student 1.3 exams - 1.5 exams - 


Source: HISD AP Exam 2015—16_07-26-16 data file 
Note: * “ppts.” means percentage points. 
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able 9. Percentage of all available STAAR Level II Satisfactory and 


Level Ill Advanced scores earned for AVID and Non-AVID 
students by grade level and subject, Spring 2016 
Subject N Met Level Il* Met Level III** 
Grade Level, 6 


AVID 515 53.2 6.2 

Reading Non-AVID 3,261 49.7 9.0 

AVID 515 64.5 8.3 

Math Non-AVID 3,257 62.2 10.3 
Grade Level, 7 

AVID 376 64.1 14.1 

Reading Non-AVID 2,525 60.2 14.1 

AVID 370 60.5 13.8 

Math Non-AVID 2,436 62.2 14.6 

AVID 379 60.2 6.9 

Writing Non-AVID 2,507 59.3 9.6 
Grade Level, 8 

AVID 485 71.1 6.8 

Reading Non-AVID 2,244 63.1 9.4 

AVID 395 64.6 5.3 

Math Non-AVID 1,990 58.8 8.6 

AVID 483 59.0 7.2 

Science Non-AVID 2,218 55.5 9.6 

AVID 483 45.5 5.2 

Social Studies _Non-AVID 2,266 45.6 9.8 


Source: Cognos Chancery Ad Hoc, STAAR 3-8, July 11, 2016 

Note: *Percent Met Level II, Satisfactory 2016 Progression Standard; **Percent Met Level Ill, 
Advanced Performance Standard. STAAR English 2015-2016 Level II* Performance Standards 
were reading grade 6 (1517), grade 7 (1567), grade 8 (1587); math grade 6 (1536), grade 7 
(1575), grade 8 (1595); writing grade 7 (3550), science grade 8 (3550), social studies grade 8 
(3550). Level IIl* 2015-2016 Advanced Performance Standards were reading grade 6 (1718), 
grade 7 (1753), grade 8 (1783); math grade 6 (1772), grade 7 (1798), grade 8 (1854); writing 
grade 7 (4602), science grade 8 (4406), social studies grade 8 (4268). 


able 10. Percentage of available STAAR EOC Level Il Satisfactory and Level 


lll Advanced scores earned for AVID and Non- AVID students 
by subject, Spring 2016 


Subject N Met Level Il* Met Level III** 
AVID 167 87.4 12.6 
English | | Non-AVID 935 32.3 0.5 
AVID 163 92.6 8.6 
English Il | Non-AVID 595 31.1 0.2 
AVID 179 93.3 36.3 
Algebra! — Non-AVID 1,127 65.7 23.1 
AVID 161 98.1 28.6 
Biology Non-AVID 803 70.1 8.5 
U.S. AVID 117 99.1 23.1 
History Non-AVID 376 70.2 2.9 


Source: Cognos Chancery Ad Hoc, STAAR EOC, July 27, 2016 
Note: *Percent Met Level II, Satisfactory Student Standard; **Percent Met Level III, Advanced 
Performance Standard. 
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able 11. Propensity Score Matching results for AVID and Non-AVID students’. mean course 


grades by course type, 2015-2016 


AVID Non-AVID 
Mean Mean Mean Std. 1-tailed 
N Grade N Grade | Difference Error t p 
Grades, 6-9 
Unmatched 4A9 83.31 2,605 84.31 -1.00 389 -2.57 .005* 
Pre-AP Matched 83.31 83.38 -0.08 674 -0.11 .456 
Grades, 10-12 
Unmatched 58 82.18 189 79.35 2.83 1.460 1.94 .027* 
Pre-AP Matched 82.03 80.19 1.83 3.569 0.51 .305 
Unmatched A7 77.19 276 76.32 0.88 1.879 0.47 .319 
AP Matched 77.19 76.43 0.77 8.552 0.09 .464 
Unmatched 70.00 73.52 -3.52 8.580 -0.41 .343 
Dual Credit Matched 5 70.00 22 74.60 -4.6 11.445 -0.40 .346 


Source: PEIMS 15-16; Chancery SMS Database Historical Grades File, July 7, 2016 

Note: “Included in the analyses were students who were enrolled on October 30, 2015 (i.e., PEIMS inclusion , had an 
average daily attendance rate greater than zero, and who had spring 2015 STAAR Reading scores (2015-2016 
grades 6-9) or 2014—2015 cumulative GPA (2015-2016 grades 10-12) only. Students were matched on economic, 
at-risk, special education, gifted/talented, limited English proficiency statuses, and their prior year’s cumulative GPA 
(2015-2016 grades 10—12) or spring 2015 STAAR performance (2015-2016 grades 6-9). *Indicates p<0.05. 


able 12. Results for estimated average AVID treatment effects on course grades for AVID 


students by grade levels and course type, 2015-2016 


Program Al Robust [95% Confidence 
Effects Coefficient Standard Error z P>|z | Interval] 
Grades, 6-9 
Pre-AP AVID/ATET .0321261 .4305192 0.07 0.941 -.811676 .8759283 
Grades, 10-12 
Pre-AP AVID/ATET 2.427603 1.01994 2.38 0.017* 4285566 4.426649 
AP AVID/ATET  -.3029529 1.329679 -0.23 0.820 -2.909076 2.30317 
Dual Credit AVID/ATET -.9602217 1.398341 -0.69 0.492 -3.70092 1.780477 


Source: PEIMS 15-16; Chancery SMS Database Historical Grades File, July 7, 2016 

Note: *Indicates statistically significant p<0.05. Grade levels with fewer than five AVID students were not matched 
with non-AVID students and were not included in the analyses. Program/Treatment-effects — AVID/ATET indicates 
the estimated average treatment effect among AVID students. 
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able 13. Propensity Score Matching results for identified AVID and non-AVID students’ by 


grade level and STAAR subject, 2015-2016 


AVID Non-AVID 
Mean Mean 
Scale Scale Mean 1-tailed 
N Score N Score Difference Std. Error t p 
Readin 
Grades Unmatched 1527.38 1529.99 -2.61 7.18 -0.36 .359 
Matched 395 1527.38 | 2,475 1510.06 17.32 10.51 1.65 .050* 
Grade 7 Unmatched 1618.64 1610.70 7.94 7.73 1.03 .152 
Matched 327 1616.72 | 2,008 1622.23 -5.51 15.43 -0.36  .359 
Grades Unmatched 1638.74 1630.71 8.03 6.70 1.20 .115 
Matched 406 1638.74 | 1,785 1648.19 -9.45 10.89 -0.87 .192 
Math 
Grade 6 Unmatched 1593.99 1601.40 -7.41 7.40 -1.00 .159 
Matched 390 1593.99 | 2,417 1586.21 7.78 11.58 0.67 .251 
Grade 7 Unmatched 1642.96 1646.73 -3.77 8.97 -0.42 337 
Matched 319 1639.82 | 1,859 1661.60 | -21.79 15.69 -1.39 .082 
Grades Unmatched 1655.75 1631.02 | 24.73 7.72 3.20 .0007* 
Matched 299 1655.75 | 1,364 1643.77] 11.99 15.94 0.75 227 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, STAAR 3-8, July 11, 2016. 


Note: “Included in the analyses were students who were enrolled on October 30, 2015 (i.e., PEIMS inclusion), had an 
average daily attendance rate greater than zero, and who had spring 2014 and spring 2015 STAAR reading or math 
scores (2015-2016 grades 6-8) only. Students were matched on the number of years in which their school 
participated in the AVID program, their economic and at-risk statuses as well as their special education, 
gifted/talented, limited English proficiency statuses, and their spring 2015 STAAR reading or math scores. *Indicates 
statistically significant p<0.05. Writing, Science, and Social Studies data were not sufficient for analyses due to 
violations of the treatment overlap assumption or tests for collinearity. 


able 14. Results for estimated average AVID treatment effects on STAAR scale scores for AVID 


students by grade level and subject, 2015-2016 


Program Al Robust [95% Confidence 
Effects Coefficient Standard Error z P> |z | Interval] 
Reading 
Grade6 AVID/ATET 11.43187 5.201527 2.20 0.028* 1.237068 21.62668 
Grade 7  AVID/ATET  12.73216 8.319905 1.53 0.126  -3.574555 ~=29.03887 
Grade 8  AVID/ATET -4.138197 5.652131 -0.73 0.464 -15.21617 6.939777 
Math 
Grade 6  AVID/ATET 7.614287 4.857901 1.57 0.117 =-1.907024 17.1356 
Grade 7 AVID/ATET  -4.207308 7.358499 -0.57 0.567 -18.6297 —10.21508 
Grade 8  AVID/ATET 14.34916 7.782431 1.84 0.065 -.904123 — 29.60244 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, STAAR 3-8, July 11, 2016. 
Note: *Indicates statistically significant p<0.05. Program/Treatment-effects — AVID/ATET indicates the estimated 


average treatment effect among AVID students. 
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able 15. Propensity Score Matching results for identified AVID and Non-AVID students’ by 


STAAR EOC subject, Spring 2016 


AVID Non-AVID 
Mean Mean 
Scale Scale Mean 1-tailed 
Subject N Score N Score [Difference Std. Error t p 
Unmatched 3575.94 3446.46 | 129.48 105.28 1.23 .110 
English | Matched 17 3575.94 480 3463.35 | 112.59 135.58 0.83 .203 
Unmatched 3751.35 3442.42 | 308.93 60.13 5.14 <.001* 
English Il Matched 48 3751.35 751 3713.54 37.81 96.852 0.39  .348 
Unmatched - - - - - - 
Algebra! Matched - - - - - - - - 
Unmatched - - - - - - 
Biology Matched - - - - - - - - 
U.S. | Unmatched 3498.00 3533.87 | -35.87 105.47 -0.34 .367 
History Matched 7 3498.00 283 3660.57 | -162.57 233.48 -0.70 242 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, STAAR EOC, July 27, 2016. 

Note: “Included in the analyses were only students who were enrolled on October 30, 2015 (i.e., PEIMS inclusion), 
had an average daily attendance rate greater than zero, and who had a spring 2014—2015 cumulative GPA. 

Students were matched on the number of years in which their school participated in the AVID program, their 
economic and at-risk statuses as well as their special education, gifted/talented, limited English proficiency statuses, 
and their spring 2015 GPA.*Indicates statistically significant p<0.05. Algebra | and Biology data were not sufficient for 
analyses due to data limitations. 


able 16. Results for estimated average AVID treatment effects on STAAR EOC scale scores for 


AVID students by subject, Spring 2016 


Program Al Robust [95% Confidence 
Subject _ Effects _ Coefficient Standard Error z P> |z | Interval] 
English | AVID/ATET -132.0756 138.4909 -0.95 0.340  -403.5128 139.3615 
English Il AVID/ATET 132.3161 74.19221 1.78 0.075  -13.09801 277.7301 
Algebral AVID/ATET - - : - : : 
Biology _AVID/ATET : - - : : : 
U.S. History AVID/ATET  -79.14286 62.62593 -1.26 0.206  -201.8874  43.60171 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, STAAR EOC, July 27, 2016. 

Note: *Indicates statistically significant p<0.05. Program/Treatment-effects AVID/ATET indicates the estimated 
average treatment effect among AVID students. Algebra | and Biology data were not sufficient for analyses due to 
violations of the treatment overlap assumption or tests for collinearity. Algebra | and Biology data were not sufficient 
for analyses due to data limitations. 
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able 17. Propensity Score Matching results for identified AVID and non-AVID students’ by 


grade level and PSAT 8/9 subject, 2015-2016 


AVID Non-AVID 
Mean Mean 
Scale Scale Mean 1-tailed 
N Score N Score Difference Std.Error t p 
EBRW 
Grade 8 Unmatched 371.84 371.55 29 3.63 0.08 .468 
Matched 386 371.84 1,628 369.66 2.18 10.02 0.22 413 
Grade 9 Unmatched 385.15 356.39 28.76 11.63 2.47  .007* 
Matched 33 385.15 463 379.39 5.76 16.05 0.36 .360 
Math 
Gisies Unmatched 385.23 389.47 -4.24 4.10 -1.03 .152 
Matched 386 385.23 1,628 405.10 -19.87 10.34 -1.92  .028* 
Grade 9 Unmatched 377.58 365.16 12.41 11.95 1.04 .149 
Matched 33 377.58 463 395.76 | -18.18 18.16 -1.00 .159 
Total Score 
Grades Unmatched 757.07 761.03 -3.95 6.93 -0.57 = .284 
Matched 386 757.07 1,628 774.77 -17.69 18.42 -0.96 .169 
Grade 9 Unmatched 762.73 721.56 41.17 20.06 2.05 .020* 
Matched 33 762.73 463 775.15 -12.42 28.59 -0.43  .334 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, HISD PSAT89 Fall 2015 data file, Feb. 26, 2016. 

Note: “Included in the analyses were students who were enrolled on October 30, 2015 (i.e., PEIMS inclusion), had 
an average daily attendance rate greater than zero, and who had spring 2014 and spring 2015 STAAR reading or 
math scores (2015-2016 grades 6-9) only. Students were matched on their economic and at-risk statuses as well as 
their special education, gifted/ talented, limited English proficiency statuses, and their spring 2015 STAAR reading or 
math scores. *Indicates statistically significant (p<0.05). 


able 18. Results for estimated average AVID treatment effects on PSAT 8/9 scale scores for AVID 


students by grade level and subject, 2015-2016 


Program Al Robust [95% Confidence 
Effects Coefficient Standard Error z P> |z | Interval] 
EBRW 
Grade 8 AVID/ATET -4.02 2.689814 -1.49 0.135 -9.287628 1.256251 
Grade 9 AVID/ATET 2.709348 10.44614 0.26 0.795  -17.76471 = 23.18341 
Math 
Grade 8 AVID/ATET  -7.222386 3.432295 -2.10 0.035*  -13.94956 -.4952114 
Grade 9  AVID/ATET -2.327167 13.66988 -0.17 0.865 = -29.11963 24.4653 
Total Score 
Grade 8  AVID/ATET -11.23807 4.898268 -2.29 0.022*  -20.8385  -1.637645 
Grade 9  AVID/ATET .3821809 19.63725 0.02 0.984  -38.10612  38.87049 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, HISD PSAT89 Fall 2015 data file, Feb. 26, 2016. 
Note: *Indicates statistically significant p<0.05. Program/Treatment-effects — AVID/ATET indicates the estimated 
average treatment effect among AVID students. 
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able 19. Propensity Score Matching results for identified AVID and non-AVID students’ by 


grade level and PSAT NMSQT subject, 2015-2016 


AVID Non-AVID 
Mean Mean 
Scale Scale Mean 1-tailed 
N Score N Score Difference Std. Error t p 
EBRW 
Graded Unmatched 402.00 374.53 27.47 12.78 2.15 .016* 
Matched 30 402.00 254 393.67 8.33 22.74 0.37 356 
Grade 11 Unmatched 391.67 385.32 6.35 15.90 0.40 .345 
Matched 24 391.67 220 416.25 -24.58 25.95 -0.95  .172 
Math 
Gadeio Unmatched 433.33 406.38 26.96 12.14 2.22 .014* 
Matched 30 433.33 254 417.67 15.67 21.98 0.71  .239 
Grae Unmatched 417.5 413.82 3.68 15.91 0.23 .409 
Matched 24 417.5 220 473.33 -55.83 28.55 -1.96 .026* 
Total Score 
Grade 10 Unmatched 835.33 780.91 54.43 21.18 2.57  .005* 
Matched 30 835.33 254 811.33 24.00 40.17 0.60 .274 
Grade 11 Unmatched 809.17 799.14 10.03 28.49 0.35 .363 
Matched 24 809.17 220 889.58 -80.42 49.19 -1.63 .052 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, HISD PSAT Fall 2015 data file, Feb. 26, 2016. 
Note: *Indicates p<0.05. “Included in the analyses were students who were enrolled on October 30, 2015 (i.e., 
PEIMS inclusion, had an average daily attendance rate greater than zero, and who had spring 2015 cumulative GPA. 
Students were matched on their economic and at-risk statuses as well as their special education, gifted/ talented, 
limited English proficiency statuses, and their spring 2015 GPA. 


able 20. Results for estimated average AVID treatment effects on PSAT NMSQT scale scores for 


AVID students by grade level and subject, 2015-2016 


Program Al Robust [95% Confidence 
Effects Coefficient Standard Error z P> |z Interval] 
EBRW 
Grade 10 AVID/ATET 7.555556 18.07922 0.42 0.676 -27.87907 42.99018 
Grade 11 AVID/ATET -24.79167 18.54432 -1.34 0.181  -61.13786 11.55452 
Math 
Grade 10 AVID/ATET  13.55556 17.62017 0.77 0.442  -20.97934  48.09045 
Grade 11 AVID/ATET -53.125 24.44746 -2.17 0.030* -101.0411 5.208857 
Total Score 
Grade 10 AVID/ATET  21.11111 31.02186 0.68 0.496  -39.69061  81.91284 
Grade 11 AVID/ATET -77.91667 39.14138 -1.99 0.047* -154.6324 1.200973 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, HISD PSAT Fall 2015 data file, Feb. 26, 2016. 
Note: *Indicates statistically significant result (9<=.05). Program/Treatment-effects — AVID/ATET indicates the 
estimated average treatment effect among AVID students. 
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able 21. Propensity Score Matching results for identified AVID and Non-AVID students’ by 


SAT subject, 2015-2016 


AVID Non-AVID 
Mean Mean 
Scale Scale Mean 1-tailed 
Subject N Score N Score [Difference Std. Error t p 
Critical Unmatched 400.91 363.46 37.45 17.50 2.14 .017* 
Reading Matched 22 400.91 292 396.36 4.55 24.15 0.19* 425 
Unmatched 384.55 348.18 36.36 16.04 2.27 = .012* 
Writing Matched 22  —-384.55 292 378.18 6.36 19.85 0.32* _.375 
Math- Unmatched 395.91 361.20 | 34.71 19.14 1.81 .036* 
ematics _ Matched 22 _—395.91 292: 380.91 15.00 26.66 0.56 .288 
Total Unmatched 1181.36 1072.84 | 108.52 46.31 2.34  .010* 
Score _ Matched 22 1181.36 292 1155.45 | 25.91 62.26 0.42* _.337 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, HISD SAT 2015-2016 data file, Oct. 21, 2016. 

Note: “Included in the analyses were only students who were enrolled on October 30, 2015 (i.e., PEIMS inclusion), 
had an average daily attendance rate greater than zero, and who had a spring 2014—2015 cumulative GPA. 
*Indicates p<0.05. Grade levels with fewer than five AVID or non-AVID students were not matched their counterparts 
and were not included in the analyses. Algebra | and Biology data were not sufficient for analyses due to violations of 
the treatment overlap assumption and tests for collinearity. 


able 22. Results for estimated average AVID treatment effects on SAT scale scores for AVID 


students, 2015-2016 


Program Al Robust [95% Confidence 
Subject _ Effects _ Coefficient Standard Error z P> |z | Interval] 
Critical 
Reading AVID/ATET 4.545455 20.15632 0.23 0.822  -34.96022 44.05113 
Writing AVID/ATET 6.363636 18.71098 0.34 0.734 -30.30921 43.03648 
Mathematics AVID/ATET _15.00000 19.97048 0.75 0.453 -24.14142 54.14142 
Total AVID/ATET —25.90909 45.92275 0.56 0.573 — -64.09785 115.916 


Source: PEIMS 15-16; Cognos Chancery Ad Hoc, HISD SAT 2015-2016 data file, Oct. 21, 2016. 

Note: *Indicates statistically significant p<0.05. Program/Treatment-effects AVID/ATET indicates the estimated 
average treatment effect among AVID students. Algebra | and Biology data were not sufficient for analyses due to 
violations of the treatment overlap assumption and tests for collinearity. 


HISD Research and Accountability 42 


